Many computing devices, such as smartphones, desktops, laptops, tablets, game consoles, and the like, utilize automatic speech recognition (ASR) for performing a number of tasks including voice search and short message dictation. In an effort to improve the accuracy of ASR, the use of deep neural networks (DNNs) has been proposed. DNNs are artificial neural networks with more than one hidden layer between input and output layers and may model complex non-linear relationships. DNN-derived features in Gaussian Mixture Model-Hidden Markov Model (GMM-HMM) speech recognition systems are also utilized to improve ASR performance. DNNs however, suffer from a number of drawbacks when utilized with Context Dependent-Deep Neural Network-Hidden Markov Model (CD-DNN-HMM ASR) systems as well as GMM-HMM ASR systems with DNN-derived features. These drawbacks include the inability to combine scores associated with CD-DNN-HMM and GMM-HMM with DNN-derived feature systems to further improve the accuracy of ASR. Furthermore, there are large computational costs associated with the use of DNNs as well as the current use of Principal Component Analysis (PCA) which, when utilized for ASR feature dimension reduction, results in less than optimal speech recognition accuracy. It is with respect to these considerations and others that the various embodiments of the present invention have been made.